Three-dimensional image processing unit and computer readable recording medium storing three-dimensional image processing program

ABSTRACT

A 3D image processing unit for applying a specified image processing to display a model located in a simulated 3D space on a monitor after rendering processing. The 3D image processing unit includes a memory unit for storing a rendered model image as a collection of pixel data and distance information from a viewing point of a simulated camera in correspondence with respect to said pixel data; image processing unit for applying semitransparent processing to each pixel data read from the memory unit; rewriting unit for rewriting the pixel data with said distance information from the viewing point of the simulated camera that is a specified reference value or larger; and control unit for causing the image processing unit and the rewriting unit to repeatedly operate a specified number of times while successively increasing the specified reference value and successively increasing a degree of transparency.

[0001] The present invention relates to a three-dimensional (3D) imageprocessing technique of applying a specified image processing to displaya model located in a simulated 3D space on a monitor after rendering it,which technique is applied, for example, to video game apparatuses.

BACKGROUND OF THE INVENTION AND RELATED ART STATEMENT

[0002] A hardware capable of displaying a 3D polygon with texturemapping and using semitransparent processing and fogging as specialfunctions has been conventionally known. The semitransparent polygonprocessing is used, when a polygon is to be pictured, to exhibit such aneffect of making an object model behind look through the polygonpictured by mixing a color data of a frame buffer and that of thepolygon to be pictured at a desired rate for each pixel position.Further, fogging is used to exhibit such an effect of fogging an objectby calculating a distance between an object model to be pictured and aviewing point of a simulated camera, setting a ratio of colors such thata mixture of these colors approaches a designated color according to thedistance, and mixing these colors.

[0003] In photographing by a camera, a depth of field exists and objectsmore distant than the depth of field about a focusing position areblurred. If such a phenomenon is applied to a simulated 3D space such asa game space, a better presence of attendance can be provided. If it isapplied to a video game, a better cubic effect can be provided and apossibility of providing a game using the presence of a depth of fieldcan be given. However, this could not be realized by the conventionalsemitransparent processing and fogging.

SUMMARY OF THE INVENTION

[0004] In view of the above situation, an object of the presentinvention is to provide an image processor for processing a 3D imagesuch that an effect of exhibiting a depth of field can be easilypresented when an object model located in a simulated 3D space isdisplayed on a monitor, and a readable recording medium storing a 3Dimage processing program.

[0005] In order to fulfill the above object, according to the presentinvention, 3D image processing unit for applying a specified imageprocessing to display a model located in a simulated 3D space on amonitor after rendering processing. The 3D image processing unit,according to the present invention comprises a memory unit for storing arendered model image as a collection of pixel data and distanceinformation from a viewing point of a simulated camera in correspondencewith respect to said pixel data; image processing unit for applyingsemitransparent processing to each pixel data read from the memory unit;rewriting unit for rewriting the pixel data with said distanceinformation from the viewing point of the simulated camera that is aspecified reference value or larger; and control unit for causing theimage processing unit and the rewriting unit to repeatedly operate aspecified number of times while successively increasing the specifiedreference value and successively increasing a degree of transparency.

[0006] With the above 3D image processing unit, each of the pixel datais stored at corresponding pixel location and is readable from thememory unit and the read out pixel data is subject to thesemitransparent processing and distance information with respect to eachof the pixel data is stored in the memory unit. The distance informationwith respect to each of the pixel data is judged as to whether it isequal or larger than the specified reference value. Then the pixel datathat underwent semitransparent processing having the correspondingdistance information equal to or larger than the specified referencevalue is rewritten to at the same pixel position in the memory unit. Thesemitransparent processing is repeated by the specified number of timeswhile the specified reference value being increased and the degree oftransparency being increased successively. Accordingly, the imagedisplayed in the simulated 3D space is given a great stage effect of thedepth of field processing.

[0007] These and other objects, features and advantages of the presentinvention will become more apparent upon a reading of the followingdetailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008]FIG. 1 is a block diagram showing one embodiment of a gameapparatus to which a 3D image processing unit according to the presentinvention is applied,

[0009]FIG. 2 is an exemplary block diagram for the execution of a depthof field processing,

[0010]FIGS. 3A, 3B and 3C are diagrams showing image examples used toexplain the depth of field processing, wherein FIG. 3A shows a screenbefore the processing, FIG. 3B shows a screen after completion of afirst loop, and FIG. 3C shows a screen after completion of a secondloop, and

[0011]FIG. 4 is a flow chart showing a procedure of the depth of fieldprocessing.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS OF THE PRESENTINVENTION

[0012]FIG. 1 is a block diagram showing one embodiment of a gameapparatus to which a 3D image processing unit according to the presentinvention is applied.

[0013] In FIG. 1, a game apparatus 1 is provided with a main game unit(not shown), a monitor 2 for displaying images of a game, a mainpreamplifier 3 and a loudspeaker 4, and a recording medium 5 storinggame data including image data, sound data and program data is mountableif necessary. The recording medium 5 may be built in or detachablymountable into the main game unit, and a ROM cassette, an optical disk,a flexible disk may be used as the recording medium 5 in addition to aROM in which a game program is stored.

[0014] Operations in the main game unit and a control system arerealized by connecting a RAM 8 for temporarily storing various data, aROM 9 for storing programs such as an operation system, interfacecircuits 10, 11, 14, 15, a signal processor 12 and an image processor 13with a CPU 6 as a central processing unit for controlling the respectiveelements via buses 7 including address busses, data buses and controllerbuses. In the case of a detachably mountable recording medium 5, variousdata in the CPU 6 are read and written in the RAM 8 all at once orsuccessively if necessary with the recording medium 5 mounted in themain game unit.

[0015] The RAM 8 includes a frame buffer 81 having a storage capacitycorresponding to the number of pixels forming at least one screen andadapted to temporarily store an image to be displayed on the monitor 2,a Z-value memory 82 for storing a distance information of a pixel datacorresponding to each pixel of the image stored in the frame buffer 81,and a data memory 83 for storing various parameters necessary for adepth of field processing and other necessary data. The ROM 9 is adaptedto store a program of an operation system and the like.

[0016] The signal processor 12 performs calculation of positions ofobject models such as a character and a fixed object in a simulated 3Dspace and calculation of a moved position of a viewing point (position)of a simulated camera, generation and processing of sound data, and isprovided with a distance calculator 121 for calculating distances of therespective object models from the viewing point of the simulated camera.

[0017] The image processor 13 writes the image data to be displayed onthe monitor 2 in the frame buffer 81 pixel by pixel using the respectivecalculation results from the signal processor 12 (rendering texturemapping processing) and is provided with an address generator 131 forgiving a read/write (R/W) commands to the RAM 8 and designating thepixel of the frame buffer 81, and a depth of field processor 132 forexecuting the depth of field processing to be described later.

[0018] A digital-to-analog (D/A) converter 18 feeds a video signal fromthe interface circuit 14 to the monitor 2 after converting it into ananalog signal. The image data written in the frame buffer 81 is fed tothe monitor 2 via the interface circuit 14 and the D/A converter 18 tobe displayed thereon. A D/A converter 20 and a main preamplifier 3 feeda sound signal inputted via the interface circuit 15 to the loudspeaker4 after converting it into an analog sound signal and amplifying it.According to a gaming condition, the generated sound data is writtenusing a partial area of the RAM 8.

[0019] A controller 17 is provided with operable members such asoperation buttons and operation levers operable by a game player, andsignals corresponding to operations made to these operable members arefed to the CPU 6 via an operation information interface circuit 16 andthe interface circuit 11. In this way, a main character on the monitoris caused to make a motion, an action and a speech intended by the gameplayer, thereby proceeding the game.

[0020] The CPU 6 executes such a control as to move the viewing point(and viewing direction) of the simulated camera during the game inaccordance with an instruction given from the controller 17 and/or thegame program. In this way, the main character or the like can be sodisplayed as not to move out of the screen of the monitor 2, preferablydisplayed substantially in the center and in a suitable size.

[0021]FIG. 2 is an exemplary block diagram for the execution of thedepth of field processing. The depth of field processing is performed tomake the game image displayed on the monitor 2 gradually more blurred asa distance from the viewing point of the simulated camera increases,i.e. as the position is more distant in depth direction (z-direction)and to display the images of the respective pixels in such a manner asto gradually merge into (harmonize with) a surrounding color.

[0022] The CPU 6 is provided with a parameter value initializing portion61 for initializing various parameter values used in the depth of fieldprocessing, a parameter value renewing portion 62 for successivelyswitching the parameter values every time a loop processing to bedescribed later is completed, and a processing number administeringportion 63 for administering the number of the performed loopprocessings.

[0023] The image processor 132 is provided with a switch 132 a forselectively switching an output direction of a pixel data at each pixelposition read from the frame buffer 81, a filter 132 b for averaging thepixel data, a semitransparent processing device 132 c for applyingsemitransparent processing to the pixel data, and a comparing circuit132 d for comparing the distance pixel by pixel.

[0024] The switch 132 a switches the data output directions so that thepixel data are outputted to the filter 1 32 b during the depth of fieldprocessing while being introduced to the interface 14, i.e. to themonitor 2 after the depth of field processing. The filter 132 bassociates data of eight pixels around an object pixel, i.e. above,below, at the left and right and oblique from the object pixel andaverages (adds) using a digital filter (for example, coefficients forthe respective pixels are all set at {fraction (1/9)} or alternativelyweighted averaging system in which coefficients are differed may beadopted). This averaging enables the object pixel to be merged into thesurroundings. The semitransparent processing device 132 c is constructedby hardware for executing, for example, an operation: (G2−G1)×a+G1,where G2, G1 and “a” denote a pixel data after filtering, a pixel databefore filtering and a parameter of a transparency value. Execution ofthe semitransparent processing by hardware is more suited to ahigh-speed processing and can sufficiently follow a rewriting cycle(frame cycle) of the game image. Besides using the hardware, processingby software may be performed provided that a specified high-speedprocessing is possible.

[0025] The comparing circuit 132 d outputs a rewrite command signal tothe R/W address generator 131 if the distance Z from the viewing pointof the simulated camera is equal to or longer than a predeterminedreference (depth) value “z” and causes the pixel data of the objectpixel processed by the semitransparent processing device 132 c to berewritten at the same pixel position of the frame buffer 81. Thecomparing circuit 132 d is provided with a timing adjusting function,e.g. a delay circuit so that the Z-value of the pixel positioncorresponding to the object pixel having been subjected to thesemitransparent processing is synchronously discriminated.Alternatively, an address control may be executed so that the Z-value ofthe object pixel is read from the Z-value memory 82 in consideration ofa delay period.

[0026] The address generator 131 successively designates the addressesfrom the leading address to the last address to the frame buffer 81 tosuccessively read the pixel data, and performs a processing to rewriteor not to rewrite the pixel data according to the comparison result,thereby executing one loop processing. It should be noted that thecomparing circuit 132 d and the address generator 131 fulfill thefunction of the rewriting means.

[0027] As parameters used in the depth of field processing, an initialtransparency value “a”, a transparency added value “aa”, an initialreference depth value “z”, a reference depth added value “za” and thenumber “n” of the loop processing (loop processing number) are preparedin this embodiment.

[0028] The parameter value initializing portion 61 sets the initialtransparency value “a” and the initial reference depth value “z” to aninitial value 0 and the loop processing number “n” to “n0” to beadministered every screen rewriting cycle (frame cycle) to the monitor2. The parameter value renewing portion 62 adds the transparency addedvalue “aa” to the present transparency value “a”, adds the referencedepth added value “za” to the present reference depth value “z” andincrements the loop processing number “n” by 1 every time one loopprocessing is performed. The processing number administering portion 63monitors the loop processing number “n” and outputs an end signal tocomplete the depth of field processing when the processing number “n”reaches the predetermined value “n0”.

[0029]FIGS. 3A, 3B and 3C are diagrams showing image examples used toexplain the depth of field processing, wherein FIG. 3A shows a screenbefore the processing, FIG. 3B shows a screen after completion of afirst loop, and FIG. 3C shows a screen after completion of a secondloop. It should be noted that the processing number “n0” to beadministered is at least two or larger.

[0030] As shown in FIG. 3A, a sphere model M10, a cone model M20 and arectangular parallelepiped model M30 are displayed on a screen 21 of themonitor 2 while being successively aligned in a direction more away fromthe viewing point of the simulated camera. The sphere model M10 islocated within a distance z1, the cone model M20 is located within adistance range of z1 to z2, and the rectangular parallelepiped model M30is located farther than the distance z2 from the viewing point of thesimulated camera.

[0031] For the sake of convenience, description is made assuming thatreference depths for the first and second loops are z1, z2,respectively. Upon completion of the first loop processing, the spheremodel M10 remains unchanged, but the cone model M20 and the rectangularparallelepiped model M30 have been changed into a cone model M21 and arectangular parallelepiped model M31 (represented by wavy lines in FIG.3B) after having been subjected to filtering and semitransparentprocessing as shown in FIG. 3B. Subsequently, after completion of thesecond loop processing, the sphere model M10 still remains unchanged(see FIG. 3A), the cone model M21 remains the same as the one after thefirst loop, and only the rectangular parallelepiped model M31 has beenchanged into a rectangular parallelepiped model M32 (degree of wavinessincreased) after having been further subjected to filtering andsemitransparent processing as shown in FIG. 3C.

[0032]FIG. 4 is a flow chart showing a procedure of the depth of fieldprocessing.

[0033] Upon starting this flow chart, the initial transparency value“a”, the transparency added value “aa”, the initial reference depthvalue “z” and the reference depth added value “za” are set at theinitial value 0 and the loop processing number “n” is set at “no” (StepS1). Subsequently, the rendered model image from the image processor 13is written in the frame buffer 81 (Step S3). When the image data iswritten in the frame buffer 81, the depth of field processing isperformed. Specifically, the data of the object pixels and those oftheir neighboring pixels are synchronously read from the frame buffer 81by successively designating the pixel positions. The read data aresubjected to the semitransparent processing via the hardware forexecuting the aforementioned operation after being subjected to thefiltering, and such a processing as to rewrite the pixel data after theprocessing over the object pixel in the frame buffer 81 or not torewrite based on the presence or absence of the rewrite command signalfrom the comparing circuit 132 d is successively performed for theleading pixel through the last pixel (Step S5).

[0034] Upon confirming the completion of the first loop processing, theparameters are changed (renewed). Specifically, the transparent addedvalue “aa” is added to the initial transparency value “a”, the referencedepth added value “za” is added to the initial reference depth value“z”, and the loop processing number “n” is incremented by 1 (Step S7).Then, it is discriminated whether the loop processing number “n” is lessthan the loop processing number “n0” to be administered (Step S9).

[0035] If the discrimination result in Step S9 is affirmative, thisroutine returns to Step S3 to execute Steps S5, S7 for the second loopusing the new transparency value “a” and reference depth value “z”. Theloop processing is performed until the discrimination result becomesnegative in Step S9. Thus, in the second loop, the transparency addedvalue “aa” is added to the first transparency value “a” and thereference depth added value “za” is added to the reference depth value“z”, with the result that the present transparency value “a” and thepresent reference depth value “z” are successively increased as moreloop processings are performed. In this way, the presentation effects ofthe blurred images and the images having been subjected to thesemitransparent processing become exponential as in the case of theactual depth of field. On the other hand, if the discrimination resultin Step S9 is negative (NO in Step S9), this flow ends. Thereafter, theimage display processing is started, i.e. the CPU 6 causes the switch132 a to be switched to the monitor 2.

[0036] As described above, the filtering processing and thesemitransparent processing are substantially enabled only by the memorycapacity of the frame buffer 81, and the model images can be graduallyblurred and merge into and harmonize with the surrounding colors indepth direction by a simple construction and a high-speed processing.

[0037] The present invention may be embodied as follows.

[0038] (1) Although the transparency added value “aa” and the referencedepth added value “za” are fixed values in the foregoing embodiment, thetransparency added value “aa” and/or the reference depth added value“za” may be individually so set at suitable values as to correspond tothe loop processing number, e.g. by means of correspondence in a tableformat. In such a case, presentation effects of a desired blurrednessand a desired semitransparent processing can obtained. Particularly,possibility and versatility can be further increased if the blurrednessand the semitransparent processing can be changed according to needs.

[0039] (2) Instead of constructing the depth of field processing device132 by hardware, it may be realized by software. In such a case, aprocessing program in accordance with the processing procedure of thehardware may be written.

[0040] (3) If the depth of field processing is performed at least onceas a loop processing, blurredness in depth direction and mergence intoand harmonization with the surroundings can be at least expressed.

[0041] (4) Although the semitransparent processing is performed usingthe pixel data having been subjected to the filtering processing in theforegoing embodiment, these two processings may be simultaneouslyperformed or performed in a reverse order. In other words, in whichsequence they are performed does not matter provided that bothblurredness and mergence into and harmonization with the surroundingscan be suitably expressed on the monitor screen. Averaging by the filter132 b may be performed using a device other than the digital filter.Further, it is not necessary to set the respective coefficients of thedigital filter at constant values. Desired coefficients may be setaccording to the aimed blurred state and semitransparent processing. Insuch a case, averaging may be performed using four (2×2) pixel data ordata of pixels arrayed in different numbers in column and row directions(e.g. 1×2, 2×3) instead of using 9 pixel data.

[0042] (5) The game apparatus according to the foregoing embodimenttakes various modes including so-called a game apparatus for businessuse, a game apparatus for home use and a general personal computer. Inthe case of the game apparatus for business use, the controller 17includes a joystick or various shot-switches as an operation inputdevice as described above, and the monitor unit includes a special CRT,liquid crystal display or like monitor 2 and a projection screen ontowhich an image is to be projected from the monitor 2. In the case of thegame apparatus for home use, the controller 17 normally includes across-shaped key and various operation buttons, the control unit and thelike are all installed in the game apparatus, and TV monitors are oftenused as the monitor. Further in the case of the personal computer, akeyboard, a mouse or like input device may be used as the controller 17,and a graphic display is used as the monitor. Further in the case of thegame apparatus for home use and the personal computer, the game programstored in the game program storage is initially stored in a floppy disk,CD-ROM, photomagnetic disk, DVD-ROM or like recording medium readable bythe computer, and is transferred to the main game unit by reading thisrecording medium by means of a reading means provided in the gameapparatus for home use and the person computer.

[0043] With the aforementioned features of the present invention, thefollowing advantages will be obtained.

[0044] The image displayed in the simulated 3D space is given a greatstage effect of the depth of filed processing with ease and relativelysimple structure. The images within the range of the specified referencedepth value are not subject to the image processing (such assemitransparent processing), thus the image displayed on the screen as awhole becomes similar to the images taken by a camera or video-camera,enabling to produce the realistic images on the screen. Moreover, thoseimages located further away from a viewer (a game player) in the depthdirection of the screen appear blurred (indistinct) than the otherimages located closer to the viewer in the depth direction of thescreen, enabling realistic display of images on the screen. Furthermore,the further the images displayed on the screen being away from theviewer, the more the images appear to fuse into the surrounding,enabling more realistic presentation of the images on the screen. Inaddition, the depth of field processing is carried out in case theviewing point of the simulated camera is moved.

[0045] This application is based on Japanese application serial no.2000-209016 filed in Japan on Jul. 10, 2000, the contents of which arehereby incorporated by reference.

[0046] As this invention may be embodied in several forms withoutdeparting from the spirit of essential characteristics thereof, thepresent embodiment is therefore illustrative and not restrictive, sincethe scope of the invention is defined by the appended claims rather thanby the description preceding them, and all changes that fall withinmetes and bounds of the claims, or equivalence of such metes and boundsare therefore intended to be embraced by the claims.

What is claimed is:
 1. A 3D image processing unit for applying aspecified image processing to display a model located in a simulated 3Dspace on a monitor after rendering processing, said 3D image processingunit comprising: a first memory for storing a rendered model image as acollection of pixel data, a second memory for storing a distanceinformation from a viewing point of a simulated camera in correspondencewith each pixel position, an image processing means for applyingaveraging and semitransparent processing to each pixel data read fromthe first memory, a rewriting means for rewriting the pixel data fromthe image processing means at the same pixel position as that of thefirst memory from which the pixel data was read when said distanceinformation from the viewing point of the simulated camera is aspecified reference value or larger, and a control means for causing theimage processing means and the rewriting means to repeatedly operateonly a specified number of times while successively increasing thespecified reference value and successively increasing a degree oftransparency, the image in the first memory being introduced to themonitor after processing by the control means is completed.
 2. A 3Dimage processing unit according to claim 1, wherein the image processingmeans applies the semitransparent processing to the averaged pixel data.3. A 3D image processing unit according to claim 1, wherein therewriting means does not rewrite the image data at the pixel positionwhere said distance information from the viewing point of the simulatedcamera is below the specified reference value.
 4. A 3D image processingunit according to claim 1, wherein the control means successivelyincreases the specified reference value by a predetermined amount.
 5. A3D image processing unit according to claim 1, wherein the control meanssuccessively increases the degree of transparency by a predeterminedamount.
 6. A 3D image processing unit according to claim 1, furthercomprising an externally operable member capable of moving the viewingpoint of the simulated camera in the simulated 3D space as it isoperated, and a distance calculating means for calculating the distancefrom the viewing point of the simulated camera for each pixel position.7. A computer-readable recording medium storing a 3D image processingprogram for applying a specified image processing to a model imagerendered from a model located in a simulated 3D space and stored in aframe buffer to display the model image to display it on a monitor, saidprocessing program comprising the steps of: storing a distanceinformation from a viewing point of a simulated camera in correspondencewith each pixel position of the model image; reading pixel data from theframe buffer and applying averaging and semitransparent processingthereto; for the pixel data at the pixel position where said distanceinformation from the viewing point of the simulated camera is aspecified reference value or larger, rewriting a storage content at thesame pixel position of the frame buffer from which the pixel data wasread; repeatedly performing the averaging and semitransparent processingand rewriting a specified number of times while the specified referencevalue is successively increased and a degree of transparency issuccessively increased; and introducing image in the frame buffer to themonitor after such processings are completed.
 8. The computer-readablerecording medium storing 3D image processing program according to claim7, wherein the semitransparent processing is applied to the averagedpixel data.
 9. The computer-readable recording medium storing 3D imageprocessing program according to claim 7, wherein the rewriting is suchthat the image data is not rewritten for the pixel position where saiddistance information from the viewing point of the simulated camera isbelow the specified reference value.
 10. The computer-readable recordingmedium storing 3D image processing program according to claim 7, whereinthe specified reference value is successively increased by apredetermined amount.
 11. The computer-readable recording medium storing3D image processing program according to claim 7, wherein the degree oftransparency is successively increased by a predetermined amount. 12.The computer-readable recording medium storing 3D image processingprogram according to claim 7, wherein the viewing point of the simulatedcamera is moved as an externally operable member is operated, and thedistance from the viewing point of the simulated camera is calculatedfor each pixel position.
 13. A 3D image processing unit for applying aspecified image processing to display a model located in a simulated 3Dspace on a monitor after rendering processing, said 3D image processingunit comprising: a memory unit for storing a rendered model image as acollection of pixel data and distance information from a viewing pointof a simulated camera in correspondence with respect to said pixel data,image processing unit for applying semitransparent processing to eachpixel data read from the memory unit, rewriting unit for rewriting thepixel data with said distance information from the viewing point of thesimulated camera that is a specified reference value or larger, andcontrol unit for causing the image processing unit and the rewritingunit to repeatedly operate a specified number of times whilesuccessively increasing the specified reference value and successivelyincreasing a degree of transparency.